Cell-type Identification
In this section, we’ll demonstrate two ways to label cells in our
dataset.
- SingleR
method, which uses correlation of gene expression
- Seurat
Integration Mapping which uses an integration method that is very
similar to the one we used to integrate our two samples.
To start, we set our library path:
LIB='/cluster/tufts/hpc/tools/R/4.0.0/'
.libPaths(c("",LIB))
We require three new packages: 1) Singler 2) celldex 3) pheatmap
suppressPackageStartupMessages({
library(tidyverse)
library(Seurat)
library(SingleR)
library(celldex)
library(pheatmap)
})
Set the base dir:
baseDir <- "~/intro_to_scrnaseq/"
We begin by loading our integrated samples.
integ_seurat = readRDS(file.path(baseDir, "data/clustered_seurat.rds"))
Set our identities to be the clusters found at the resolution 0.4 and
plot UMAP:
Idents(object = integ_seurat) <- "integrated_snn_res.0.4"
DimPlot(integ_seurat, label=T)

Set our identities to be the sample type and plot UMAP:
Idents(object = integ_seurat) <- "sample"
DimPlot(integ_seurat)
## Correlation Method
(SingleR) We’ll use the SingleR tool with a reference
database of expression profiles of known cell types in order to identify
our cells and clusters. As mentioned in the lecture, this method
measures the correlation of overall gene expression between cells in a
reference database with cells in the query dataset in order to label
cells
To start, we’ll use a general database of Human pure cell-types
called the Human Primary Cell Type Atlas. This dataset along with
several others is available through the celldex
R library. To load:
hpca = HumanPrimaryCellAtlasData()
The HPCA object is of the data type called a
Summarized Experiment which allows one to store count data
matrices in assays along with metadata which annotate each cell/sample
in the count data.
head(hpca)

Well use in particular the label.main column of the metadata, which
has the following cell-types:
unique(hpca$label.main)

Our data to be labeled is input into SingleR as a normalized count
matrix, which we can extract from the RNA assay our
integ_seurat object:
query_counts = integ_seurat@assays$RNA@data
SingleR can be run both on the cluster level and the individual cell
level. For cluster-level annotation, the average expression profile of
each cluster is used and a single label is generated. This is much
faster to run, so we’ll start here.
query_clusters = integ_seurat@meta.data$integrated_snn_res.0.4
The following command runs SingleR on the cluster level, which should
take only a few seconds.
pred_cluster <- SingleR(test = query_counts,
ref = hpca,
assay.type.test="logcounts",
clusters = query_clusters,
labels = hpca$label.main,
prune=F)
Save the results:
saveRDS(pred_cluster, file.path(baseDir, "results/singler_hpca_cluster_res0.4.rds"))
We can view the results, which contain a score for every cell type
plus the final label:
view(pred_cluster)
Select the score data to plot as a heatmap and the label
column to annotate:
scores = data.frame(pred_cluster) %>%
dplyr::select(starts_with("scores"))
labels = data.frame(pred_cluster) %>%
dplyr::select("labels")
The scores can be plotted as a heatmap:
pheatmap(scores,
annotation_row = labels)

Now, make a named list with new names:
new_names = pred_cluster$labels
names(new_names) = rownames(pred_cluster)
new_names
Set the identities to the clusters found at resolution 0.4, rename
the clusters, and add the names to the seurat metadata:
Idents(object = integ_seurat) <- "integrated_snn_res.0.4"
integ_seurat = RenameIdents(integ_seurat,
new_names)
integ_seurat$labels = Idents(integ_seurat)
Let’s look at the labeled clusters:
Idents(integ_seurat) = "labels"
DimPlot(integ_seurat,
label=T)

Running on the individual cell level will take longer, so we’ll run
it as a batch job. To do this, navigate to our scripts directory and
open singler_cell.R. This file contains the key steps
above, but eliminates the labels argument from the SingleR
command.
# DO NOT RUN!
pred_cell <- SingleR(test = query_counts,
ref = hpca,
assay.type.test="logcounts",
labels = hpca$label.main)
To run it, we use the run_singler_cell.sh script in the
scripts directory. Click to open the file:
#!/bin/bash
#SBATCH -J run_singler
#SBATCH --time=2:00:00
#SBATCH -n 1
#SBATCH -N 1
#SBATCH --mem=10Gb
#SBATCH --output=%j.out
#SBATCH --error=%j.err
module purge
module load R/4.0.0
Rscript --no-save singler_cell.R
This contains an sbatch header which gives instructions to the HPC
job scheduler, ‘slurm’, about resources that the job will need.
To run the script: - Click on Terminal next to
Console in the bottom portion of the Rstudio application -
Change to our scripts directory by typing
cd intro_to_scrnaseq/scripts - Type
sbatch run_singler_cell.sh singler_cell.R and press enter.
- Your job will be given a number by slurm and placed in the queue. - To
check the status of your job, type squeue -u tufts-username
and you will see your job status.
Let’s load the prepossessed results in the meantime:
pred_cell = readRDS(file.path(baseDir,"data/singler_hpca_cell.rds"))
Take a look at the cell level labels which should be done running by
now. We’ll this time, we’ll add the pruned labeles to the
seurat object metadata. Note we add it directly to the metadata because
it has one entry for each cell.
integ_seurat = AddMetaData(integ_seurat,
pred_cell$pruned.labels,
"hpca.labels")
Assign the idents and make a plot:
Idents(integ_seurat) = "hpca.labels"
DimPlot(integ_seurat,
label=T)

We see the picture is more complex and clusters containing a mix of
cell labels. We can view the breakdown per cluster as a heatmap:
tab <- table(cluster=integ_seurat$integrated_snn_res.0.4,
label=pred_cell$labels)
pheatmap(log10(tab+10))

Some clusters appear to have a mix of cells, which may indicate that
they contain a type of cell not in our reference database. This is
expected since we’ve used a very general database. Next we’ll use a
single-cell RNAseq dataset that contains a perfect match and see how the
labeling changes.
Integration Mapping Method (Seurat)
These are PBMC from another source, processed through the Seurat
pipeline as our data. Let’s load and view the metadata:
pbmc = readRDS(file.path(baseDir, "data/pbmc_reference.rds"))
head(pbmc)
Set the identities and plot
Idents(pbmc) = "seurat_annotations"
DimPlot(pbmc, label=T)

Find the transfer anchors:
anchors <- FindTransferAnchors(reference = pbmc,
query = integ_seurat,
reduction = "pcaproject",
reference.reduction = "pca",
dims = 1:30)
Make cell type predictions by transfering the anchors:
predictions<- TransferData(anchorset = anchors,
refdata = pbmc$seurat_annotations,
dims = 1:30)
Add the predicted id to the metadata:
integ_seurat <- AddMetaData(integ_seurat,
metadata = predictions)
Set the Idents and plot:
Idents(integ_seurat) = "predicted.id"
DimPlot(integ_seurat, label=T )

We can view the breakdown per cluster as a heatmap:
tab <- table(cluster=integ_seurat$integrated_snn_res.0.4,
label=integ_seurat$predicted.id)
pheatmap(log10(tab+10))

Finally, we have to save the labeled object:
saveRDS(integ_seurat, file.path(baseDir,"results/labeled_seurat.rds"))
LS0tCnRpdGxlOiAiQ2VsbCB0eXBlIGlkZW50aWZpY2F0aW9uIgpvdXRwdXQ6IGh0bWxfbm90ZWJvb2sKLS0tCgojIyBDZWxsLXR5cGUgSWRlbnRpZmljYXRpb24KSW4gdGhpcyBzZWN0aW9uLCB3ZSdsbCBkZW1vbnN0cmF0ZSB0d28gd2F5cyB0byBsYWJlbCBjZWxscyBpbiBvdXIgZGF0YXNldC4gCgotIFtTaW5nbGVSXShodHRwczovL2Jpb2NvbmR1Y3Rvci5vcmcvcGFja2FnZXMvZGV2ZWwvYmlvYy92aWduZXR0ZXMvU2luZ2xlUi9pbnN0L2RvYy9TaW5nbGVSLmh0bWwpIG1ldGhvZCwgd2hpY2ggdXNlcyBjb3JyZWxhdGlvbiBvZiBnZW5lIGV4cHJlc3Npb24gICAKLSBbU2V1cmF0IEludGVncmF0aW9uIE1hcHBpbmddKGh0dHBzOi8vc2F0aWphbGFiLm9yZy9zZXVyYXQvYXJ0aWNsZXMvaW50ZWdyYXRpb25fbWFwcGluZy5odG1sKSB3aGljaCB1c2VzIGFuIGludGVncmF0aW9uIG1ldGhvZCB0aGF0IGlzIHZlcnkgc2ltaWxhciB0byB0aGUgb25lIHdlIHVzZWQgdG8gaW50ZWdyYXRlIG91ciB0d28gc2FtcGxlcy4KClRvIHN0YXJ0LCB3ZSBzZXQgb3VyIGxpYnJhcnkgcGF0aDoKYGBgUgpMSUI9Jy9jbHVzdGVyL3R1ZnRzL2hwYy90b29scy9SLzQuMC4wLycKLmxpYlBhdGhzKGMoIiIsTElCKSkKYGBgCgpXZSByZXF1aXJlIHRocmVlIG5ldyBwYWNrYWdlczoKMSkgU2luZ2xlcgoyKSBjZWxsZGV4CjMpIHBoZWF0bWFwCgpgYGBSCnN1cHByZXNzUGFja2FnZVN0YXJ0dXBNZXNzYWdlcyh7CiAgbGlicmFyeSh0aWR5dmVyc2UpCiAgbGlicmFyeShTZXVyYXQpCiAgbGlicmFyeShTaW5nbGVSKQogIGxpYnJhcnkoY2VsbGRleCkKICBsaWJyYXJ5KHBoZWF0bWFwKQp9KQpgYGAKClNldCB0aGUgYmFzZSBkaXI6CmBgYFIKYmFzZURpciA8LSAifi9pbnRyb190b19zY3JuYXNlcS8iCmBgYAoKV2UgYmVnaW4gYnkgbG9hZGluZyBvdXIgaW50ZWdyYXRlZCBzYW1wbGVzLgoKYGBgUgppbnRlZ19zZXVyYXQgPSByZWFkUkRTKGZpbGUucGF0aChiYXNlRGlyLCAiZGF0YS9jbHVzdGVyZWRfc2V1cmF0LnJkcyIpKQpgYGAKClNldCBvdXIgaWRlbnRpdGllcyB0byBiZSB0aGUgY2x1c3RlcnMgZm91bmQgYXQgdGhlIHJlc29sdXRpb24gMC40IGFuZCBwbG90IFVNQVA6IApgYGBSCklkZW50cyhvYmplY3QgPSBpbnRlZ19zZXVyYXQpIDwtICJpbnRlZ3JhdGVkX3Nubl9yZXMuMC40IgpEaW1QbG90KGludGVnX3NldXJhdCwgbGFiZWw9VCkKYGBgCiFbXShpbWFnZXMvaW50ZWdyYXRlZF9jbHVzdGVyLnBuZykKClNldCBvdXIgaWRlbnRpdGllcyB0byBiZSB0aGUgc2FtcGxlIHR5cGUgYW5kIHBsb3QgVU1BUDoKYGBgUgpJZGVudHMob2JqZWN0ID0gaW50ZWdfc2V1cmF0KSA8LSAic2FtcGxlIgpEaW1QbG90KGludGVnX3NldXJhdCkKYGBgCiFbXShpbWFnZXMvaW50ZWdyYXRlZF9zYW1wbGUucG5nKQojIyBDb3JyZWxhdGlvbiBNZXRob2QgKFNpbmdsZVIpIApXZSdsbCB1c2UgdGhlIFtTaW5nbGVSXShodHRwczovL2dpdGh1Yi5jb20vTFRMQS9TaW5nbGVSKSB0b29sIHdpdGggYSByZWZlcmVuY2UgZGF0YWJhc2Ugb2YgZXhwcmVzc2lvbiBwcm9maWxlcyBvZiBrbm93biBjZWxsIHR5cGVzIGluIG9yZGVyIHRvIGlkZW50aWZ5IG91ciBjZWxscyBhbmQgY2x1c3RlcnMuIEFzIG1lbnRpb25lZCBpbiB0aGUgbGVjdHVyZSwgdGhpcyBtZXRob2QgbWVhc3VyZXMgdGhlIGNvcnJlbGF0aW9uIG9mIG92ZXJhbGwgZ2VuZSBleHByZXNzaW9uIGJldHdlZW4gY2VsbHMgaW4gYSByZWZlcmVuY2UgZGF0YWJhc2Ugd2l0aCBjZWxscyBpbiB0aGUgcXVlcnkgZGF0YXNldCBpbiBvcmRlciB0byBsYWJlbCBjZWxscyAgCgpUbyBzdGFydCwgd2UnbGwgdXNlIGEgZ2VuZXJhbCBkYXRhYmFzZSBvZiBIdW1hbiBwdXJlIGNlbGwtdHlwZXMgY2FsbGVkIHRoZSBIdW1hbiBQcmltYXJ5IENlbGwgVHlwZSBBdGxhcy4gIFRoaXMgZGF0YXNldCBhbG9uZyB3aXRoIHNldmVyYWwgb3RoZXJzIGlzIGF2YWlsYWJsZSB0aHJvdWdoIHRoZSBbY2VsbGRleF0oaHR0cHM6Ly9yZHJyLmlvL2dpdGh1Yi9MVExBL2NlbGxkZXgvbWFuL0h1bWFuUHJpbWFyeUNlbGxBdGxhc0RhdGEuaHRtbCkgUiBsaWJyYXJ5LiBUbyBsb2FkOgpgYGBSCmhwY2EgPSBIdW1hblByaW1hcnlDZWxsQXRsYXNEYXRhKCkKYGBgCgpUaGUgSFBDQSBvYmplY3QgaXMgb2YgdGhlIGRhdGEgdHlwZSBjYWxsZWQgYSBgU3VtbWFyaXplZCBFeHBlcmltZW50YCB3aGljaCBhbGxvd3Mgb25lIHRvIHN0b3JlIGNvdW50IGRhdGEgbWF0cmljZXMgaW4gYXNzYXlzIGFsb25nIHdpdGggbWV0YWRhdGEgd2hpY2ggYW5ub3RhdGUgZWFjaCBjZWxsL3NhbXBsZSBpbiB0aGUgY291bnQgZGF0YS4KCmBgYFIKaGVhZChocGNhKQpgYGAKIVtdKGltYWdlcy9oZWFkX2hwY2EucG5nKQoKV2VsbCB1c2UgaW4gcGFydGljdWxhciB0aGUgbGFiZWwubWFpbiBjb2x1bW4gb2YgdGhlIG1ldGFkYXRhLCB3aGljaCBoYXMgdGhlIGZvbGxvd2luZyBjZWxsLXR5cGVzOgoKYGBgUgp1bmlxdWUoaHBjYSRsYWJlbC5tYWluKQpgYGAKIVtdKGltYWdlcy91bmlxdWVfaHBjYS5wbmcpCgpPdXIgZGF0YSB0byBiZSBsYWJlbGVkIGlzIGlucHV0IGludG8gU2luZ2xlUiBhcyBhIG5vcm1hbGl6ZWQgY291bnQgbWF0cml4LCB3aGljaCB3ZSBjYW4gZXh0cmFjdCBmcm9tIHRoZSBgUk5BYCBhc3NheSBvdXIgYGludGVnX3NldXJhdGAgb2JqZWN0OgpgYGBSCnF1ZXJ5X2NvdW50cyA9IGludGVnX3NldXJhdEBhc3NheXMkUk5BQGRhdGEKYGBgCgpTaW5nbGVSIGNhbiBiZSBydW4gYm90aCBvbiB0aGUgY2x1c3RlciBsZXZlbCBhbmQgdGhlIGluZGl2aWR1YWwgY2VsbCBsZXZlbC4gRm9yIGNsdXN0ZXItbGV2ZWwgYW5ub3RhdGlvbiwgdGhlIGF2ZXJhZ2UgZXhwcmVzc2lvbiBwcm9maWxlIG9mIGVhY2ggY2x1c3RlciBpcyB1c2VkIGFuZCBhIHNpbmdsZSBsYWJlbCBpcyBnZW5lcmF0ZWQuIFRoaXMgaXMgbXVjaCBmYXN0ZXIgdG8gcnVuLCBzbyB3ZSdsbCBzdGFydCBoZXJlLgoKYGBgUgpxdWVyeV9jbHVzdGVycyA9IGludGVnX3NldXJhdEBtZXRhLmRhdGEkaW50ZWdyYXRlZF9zbm5fcmVzLjAuNApgYGAKClRoZSBmb2xsb3dpbmcgY29tbWFuZCBydW5zIFNpbmdsZVIgb24gdGhlIGNsdXN0ZXIgbGV2ZWwsIHdoaWNoIHNob3VsZCB0YWtlIG9ubHkgYSBmZXcgc2Vjb25kcy4gCmBgYFIKcHJlZF9jbHVzdGVyIDwtIFNpbmdsZVIodGVzdCA9IHF1ZXJ5X2NvdW50cywKICAgICAgICAgICAgICAgICAgICAgICAgcmVmID0gaHBjYSwKICAgICAgICAgICAgICAgICAgICAgICAgYXNzYXkudHlwZS50ZXN0PSJsb2djb3VudHMiLAogICAgICAgICAgICAgICAgICAgICAgICBjbHVzdGVycyA9IHF1ZXJ5X2NsdXN0ZXJzLAogICAgICAgICAgICAgICAgICAgICAgICBsYWJlbHMgPSBocGNhJGxhYmVsLm1haW4sIAogICAgICAgICAgICAgICAgICAgICAgICBwcnVuZT1GKQpgYGAKClNhdmUgdGhlIHJlc3VsdHM6CmBgYFIKc2F2ZVJEUyhwcmVkX2NsdXN0ZXIsIGZpbGUucGF0aChiYXNlRGlyLCAicmVzdWx0cy9zaW5nbGVyX2hwY2FfY2x1c3Rlcl9yZXMwLjQucmRzIikpCmBgYAoKV2UgY2FuIHZpZXcgdGhlIHJlc3VsdHMsIHdoaWNoIGNvbnRhaW4gYSBzY29yZSBmb3IgZXZlcnkgY2VsbCB0eXBlIHBsdXMgdGhlIGZpbmFsIGxhYmVsOgpgYGBSCnZpZXcocHJlZF9jbHVzdGVyKQpgYGAKClNlbGVjdCB0aGUgc2NvcmUgZGF0YSB0byBwbG90IGFzIGEgaGVhdG1hcCBhbmQgdGhlIGBsYWJlbGAgY29sdW1uIHRvIGFubm90YXRlOgpgYGBSCnNjb3JlcyA9IGRhdGEuZnJhbWUocHJlZF9jbHVzdGVyKSAlPiUKICBkcGx5cjo6c2VsZWN0KHN0YXJ0c193aXRoKCJzY29yZXMiKSkgCgpsYWJlbHMgPSBkYXRhLmZyYW1lKHByZWRfY2x1c3RlcikgJT4lCiAgZHBseXI6OnNlbGVjdCgibGFiZWxzIikKYGBgCgpUaGUgc2NvcmVzIGNhbiBiZSBwbG90dGVkIGFzIGEgaGVhdG1hcDoKYGBgUgpwaGVhdG1hcChzY29yZXMsCiAgICAgICAgIGFubm90YXRpb25fcm93ID0gbGFiZWxzKSAKYGBgCiFbXShpbWFnZXMvcHJlZF9jbHVzdGVyX3BoZWF0bWFwLnBuZykKCk5vdywgbWFrZSBhIG5hbWVkIGxpc3Qgd2l0aCBuZXcgbmFtZXM6CmBgYFIKbmV3X25hbWVzID0gcHJlZF9jbHVzdGVyJGxhYmVscwpuYW1lcyhuZXdfbmFtZXMpID0gcm93bmFtZXMocHJlZF9jbHVzdGVyKQpuZXdfbmFtZXMKYGBgCgpTZXQgdGhlIGlkZW50aXRpZXMgdG8gdGhlIGNsdXN0ZXJzIGZvdW5kIGF0IHJlc29sdXRpb24gMC40LCByZW5hbWUgdGhlIGNsdXN0ZXJzLCBhbmQgYWRkIHRoZSBuYW1lcyB0byB0aGUgc2V1cmF0IG1ldGFkYXRhOgpgYGBSCklkZW50cyhvYmplY3QgPSBpbnRlZ19zZXVyYXQpIDwtICJpbnRlZ3JhdGVkX3Nubl9yZXMuMC40IgppbnRlZ19zZXVyYXQgPSBSZW5hbWVJZGVudHMoaW50ZWdfc2V1cmF0LCAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgbmV3X25hbWVzKQppbnRlZ19zZXVyYXQkbGFiZWxzID0gSWRlbnRzKGludGVnX3NldXJhdCkKYGBgCgpMZXQncyBsb29rIGF0IHRoZSBsYWJlbGVkIGNsdXN0ZXJzOgpgYGBSCklkZW50cyhpbnRlZ19zZXVyYXQpID0gImxhYmVscyIKRGltUGxvdChpbnRlZ19zZXVyYXQsIAogICAgICAgIGxhYmVsPVQpCmBgYAohW10oaW1hZ2VzL3ByZWRfY2x1c3Rlcl9sYWJlbC5wbmcpCgpSdW5uaW5nIG9uIHRoZSBpbmRpdmlkdWFsIGNlbGwgbGV2ZWwgd2lsbCB0YWtlIGxvbmdlciwgc28gd2UnbGwgcnVuIGl0IGFzIGEgYmF0Y2ggam9iLiBUbyBkbyB0aGlzLCBuYXZpZ2F0ZSB0byBvdXIgc2NyaXB0cyBkaXJlY3RvcnkgYW5kIG9wZW4gYHNpbmdsZXJfY2VsbC5SYC4gVGhpcyBmaWxlIGNvbnRhaW5zIHRoZSBrZXkgc3RlcHMgYWJvdmUsIGJ1dCBlbGltaW5hdGVzIHRoZSBgbGFiZWxzYCBhcmd1bWVudCBmcm9tIHRoZSBTaW5nbGVSIGNvbW1hbmQuCgpgYGBSCiMgRE8gTk9UIFJVTiEKcHJlZF9jZWxsIDwtIFNpbmdsZVIodGVzdCA9IHF1ZXJ5X2NvdW50cywKICAgICAgICAgICAgICAgICAgICAgIHJlZiA9IGhwY2EsCiAgICAgICAgICAgICAgICAgICAgICBhc3NheS50eXBlLnRlc3Q9ImxvZ2NvdW50cyIsCiAgICAgICAgICAgICAgICAgICAgICBsYWJlbHMgPSBocGNhJGxhYmVsLm1haW4pCgpgYGAKClRvIHJ1biBpdCwgd2UgdXNlIHRoZSBgcnVuX3NpbmdsZXJfY2VsbC5zaGAgc2NyaXB0IGluIHRoZSBgc2NyaXB0c2AgZGlyZWN0b3J5LiBDbGljayB0byBvcGVuIHRoZSBmaWxlOgoKYGBgYmFzaAojIS9iaW4vYmFzaAojU0JBVENIIC1KIHJ1bl9zaW5nbGVyCiNTQkFUQ0ggLS10aW1lPTI6MDA6MDAgCiNTQkFUQ0ggLW4gMQojU0JBVENIIC1OIDEKI1NCQVRDSCAtLW1lbT0xMEdiCiNTQkFUQ0ggLS1vdXRwdXQ9JWoub3V0IAojU0JBVENIIC0tZXJyb3I9JWouZXJyIAogCm1vZHVsZSBwdXJnZQptb2R1bGUgbG9hZCBSLzQuMC4wCgpSc2NyaXB0IC0tbm8tc2F2ZSBzaW5nbGVyX2NlbGwuUgpgYGAKClRoaXMgY29udGFpbnMgYW4gc2JhdGNoIGhlYWRlciB3aGljaCBnaXZlcyBpbnN0cnVjdGlvbnMgdG8gdGhlIEhQQyBqb2Igc2NoZWR1bGVyLCAnc2x1cm0nLCBhYm91dCByZXNvdXJjZXMgdGhhdCB0aGUgam9iIHdpbGwgbmVlZC4gCgpUbyBydW4gdGhlIHNjcmlwdDoKLSBDbGljayBvbiBgVGVybWluYWxgIG5leHQgdG8gYENvbnNvbGVgIGluIHRoZSBib3R0b20gcG9ydGlvbiBvZiB0aGUgUnN0dWRpbyBhcHBsaWNhdGlvbgotIENoYW5nZSB0byBvdXIgYHNjcmlwdHNgIGRpcmVjdG9yeSBieSB0eXBpbmcgYGNkIGludHJvX3RvX3Njcm5hc2VxL3NjcmlwdHNgCi0gVHlwZSBgc2JhdGNoIHJ1bl9zaW5nbGVyX2NlbGwuc2ggc2luZ2xlcl9jZWxsLlJgIGFuZCBwcmVzcyBlbnRlci4gCi0gWW91ciBqb2Igd2lsbCBiZSBnaXZlbiBhIG51bWJlciBieSBzbHVybSBhbmQgcGxhY2VkIGluIHRoZSBxdWV1ZS4KLSBUbyBjaGVjayB0aGUgc3RhdHVzIG9mIHlvdXIgam9iLCB0eXBlIGBzcXVldWUgLXUgdHVmdHMtdXNlcm5hbWVgIGFuZCB5b3Ugd2lsbCBzZWUgeW91ciBqb2Igc3RhdHVzLiAKCkxldCdzIGxvYWQgdGhlIHByZXBvc3Nlc3NlZCByZXN1bHRzIGluIHRoZSBtZWFudGltZToKYGBgUgpwcmVkX2NlbGwgPSByZWFkUkRTKGZpbGUucGF0aChiYXNlRGlyLCJkYXRhL3NpbmdsZXJfaHBjYV9jZWxsLnJkcyIpKQpgYGAKClRha2UgYSBsb29rIGF0IHRoZSBjZWxsIGxldmVsIGxhYmVscyB3aGljaCBzaG91bGQgYmUgZG9uZSBydW5uaW5nIGJ5IG5vdy4gV2UnbGwgdGhpcyB0aW1lLCB3ZSdsbCBhZGQgdGhlIGBwcnVuZWQgbGFiZWxlc2AgdG8gdGhlIHNldXJhdCBvYmplY3QgbWV0YWRhdGEuIE5vdGUgd2UgYWRkIGl0IGRpcmVjdGx5IHRvIHRoZSBtZXRhZGF0YSBiZWNhdXNlIGl0IGhhcyBvbmUgZW50cnkgZm9yIGVhY2ggY2VsbC4KYGBgUgppbnRlZ19zZXVyYXQgID0gQWRkTWV0YURhdGEoaW50ZWdfc2V1cmF0LAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBwcmVkX2NlbGwkcHJ1bmVkLmxhYmVscywKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgImhwY2EubGFiZWxzIikKYGBgCgpBc3NpZ24gdGhlIGlkZW50cyBhbmQgbWFrZSBhIHBsb3Q6CmBgYFIKSWRlbnRzKGludGVnX3NldXJhdCkgPSAiaHBjYS5sYWJlbHMiCkRpbVBsb3QoaW50ZWdfc2V1cmF0LCAKICAgICAgICBsYWJlbD1UKQpgYGAKIVtdKGltYWdlcy9wcmVkX2NlbGxfbGFiZWwucG5nKQoKV2Ugc2VlIHRoZSBwaWN0dXJlIGlzIG1vcmUgY29tcGxleCBhbmQgY2x1c3RlcnMgY29udGFpbmluZyBhIG1peCBvZiBjZWxsIGxhYmVscy4gV2UgY2FuIHZpZXcgdGhlIGJyZWFrZG93biBwZXIgY2x1c3RlciBhcyBhIGhlYXRtYXA6CmBgYFIKdGFiIDwtIHRhYmxlKGNsdXN0ZXI9aW50ZWdfc2V1cmF0JGludGVncmF0ZWRfc25uX3Jlcy4wLjQsCiAgICAgICAgICAgICBsYWJlbD1wcmVkX2NlbGwkbGFiZWxzKQpwaGVhdG1hcChsb2cxMCh0YWIrMTApKSAKYGBgCgohW10oaW1hZ2VzL3ByZWRfY2VsbF9oZWF0bWFwLnBuZykKClNvbWUgY2x1c3RlcnMgYXBwZWFyIHRvIGhhdmUgYSBtaXggb2YgY2VsbHMsIHdoaWNoIG1heSBpbmRpY2F0ZSB0aGF0IHRoZXkgY29udGFpbiBhIHR5cGUgb2YgY2VsbCBub3QgaW4gb3VyIHJlZmVyZW5jZSBkYXRhYmFzZS4gVGhpcyBpcyBleHBlY3RlZCBzaW5jZSB3ZSd2ZSB1c2VkIGEgdmVyeSBnZW5lcmFsIGRhdGFiYXNlLiBOZXh0IHdlJ2xsIHVzZSBhIHNpbmdsZS1jZWxsIFJOQXNlcSBkYXRhc2V0IHRoYXQgY29udGFpbnMgYSBwZXJmZWN0IG1hdGNoIGFuZCBzZWUgaG93IHRoZSBsYWJlbGluZyBjaGFuZ2VzLgoKCiMjIEludGVncmF0aW9uIE1hcHBpbmcgTWV0aG9kIChTZXVyYXQpCgpUaGVzZSBhcmUgUEJNQyBmcm9tIGFub3RoZXIgc291cmNlLCBwcm9jZXNzZWQgdGhyb3VnaCB0aGUgU2V1cmF0IHBpcGVsaW5lIGFzIG91ciBkYXRhLiBMZXQncyBsb2FkIGFuZCB2aWV3IHRoZSBtZXRhZGF0YToKYGBgUgpwYm1jID0gcmVhZFJEUyhmaWxlLnBhdGgoYmFzZURpciwgImRhdGEvcGJtY19yZWZlcmVuY2UucmRzIikpCmhlYWQocGJtYykKYGBgCgpTZXQgdGhlIGlkZW50aXRpZXMgYW5kIHBsb3QKYGBgUgpJZGVudHMocGJtYykgPSAic2V1cmF0X2Fubm90YXRpb25zIgpEaW1QbG90KHBibWMsIGxhYmVsPVQpCmBgYAoKIVtdKGltYWdlcy9wYm1jX3JlZmVyZW5jZV9sYWJlbC5wbmcpCgpGaW5kIHRoZSB0cmFuc2ZlciBhbmNob3JzOgpgYGBSCmFuY2hvcnMgPC0gRmluZFRyYW5zZmVyQW5jaG9ycyhyZWZlcmVuY2UgPSBwYm1jLCAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBxdWVyeSA9IGludGVnX3NldXJhdCwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICByZWR1Y3Rpb24gPSAicGNhcHJvamVjdCIsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgcmVmZXJlbmNlLnJlZHVjdGlvbiA9ICJwY2EiLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGRpbXMgPSAxOjMwKQpgYGAKCk1ha2UgY2VsbCB0eXBlIHByZWRpY3Rpb25zIGJ5IHRyYW5zZmVyaW5nIHRoZSBhbmNob3JzOgpgYGBSCnByZWRpY3Rpb25zPC0gVHJhbnNmZXJEYXRhKGFuY2hvcnNldCA9IGFuY2hvcnMsIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHJlZmRhdGEgPSBwYm1jJHNldXJhdF9hbm5vdGF0aW9ucywKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBkaW1zID0gMTozMCkKYGBgCgpBZGQgdGhlIHByZWRpY3RlZCBpZCB0byB0aGUgbWV0YWRhdGE6CmBgYFIKaW50ZWdfc2V1cmF0IDwtIEFkZE1ldGFEYXRhKGludGVnX3NldXJhdCwgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIG1ldGFkYXRhID0gcHJlZGljdGlvbnMpCmBgYAoKU2V0IHRoZSBJZGVudHMgYW5kIHBsb3Q6CmBgYFIKSWRlbnRzKGludGVnX3NldXJhdCkgPSAicHJlZGljdGVkLmlkIgpEaW1QbG90KGludGVnX3NldXJhdCwgbGFiZWw9VCApCmBgYAoKIVtdKGltYWdlcy9pbnRlZ3JhdGVfbGFiZWwucG5nKQoKV2UgY2FuIHZpZXcgdGhlIGJyZWFrZG93biBwZXIgY2x1c3RlciBhcyBhIGhlYXRtYXA6CmBgYFIKdGFiIDwtIHRhYmxlKGNsdXN0ZXI9aW50ZWdfc2V1cmF0JGludGVncmF0ZWRfc25uX3Jlcy4wLjQsCiAgICAgICAgICAgICBsYWJlbD1pbnRlZ19zZXVyYXQkcHJlZGljdGVkLmlkKQpwaGVhdG1hcChsb2cxMCh0YWIrMTApKSAKYGBgCgohW10oaW1hZ2VzL2ludGVncmF0ZV9oZWF0bWFwLnBuZykKCkZpbmFsbHksIHdlIGhhdmUgdG8gc2F2ZSB0aGUgbGFiZWxlZCBvYmplY3Q6CmBgYFIKc2F2ZVJEUyhpbnRlZ19zZXVyYXQsIGZpbGUucGF0aChiYXNlRGlyLCJyZXN1bHRzL2xhYmVsZWRfc2V1cmF0LnJkcyIpKQpgYGA=